Prospective validation of the 4C prognostic models for adults hospitalised with COVID-19 using the ISARIC WHO Clinical Characterisation Protocol.

PURPOSE: To prospectively validate two risk scores to predict mortality (4C Mortality) and in-hospital deterioration (4C Deterioration) among adults hospitalised with COVID-19. METHODS: Prospective observational cohort study of adults (age ≥18 years) with confirmed or highly suspected COVID-19 recruited into the International Severe Acute Respiratory and emerging Infections Consortium (ISARIC) WHO Clinical Characterisation Protocol UK (CCP-UK) study in 306 hospitals across England, Scotland and Wales. Patients were recruited between 27 August 2020 and 17 February 2021, with at least 4 weeks follow-up before final data extraction. The main outcome measures were discrimination and calibration of models for in-hospital deterioration (defined as any requirement of ventilatory support or critical care, or death) and mortality, incorporating predefined subgroups. RESULTS: 76 588 participants were included, of whom 27 352 (37.4%) deteriorated and 12 581 (17.4%) died. Both the 4C Mortality (0.78 (0.77 to 0.78)) and 4C Deterioration scores (pooled C-statistic 0.76 (95% CI 0.75 to 0.77)) demonstrated consistent discrimination across all nine National Health Service regions, with similar performance metrics to the original validation cohorts. Calibration remained stable (4C Mortality: pooled slope 1.09, pooled calibration-in-the-large 0.12; 4C Deterioration: 1.00, -0.04), with no need for temporal recalibration during the second UK pandemic wave of hospital admissions. CONCLUSION: Both 4C risk stratification models demonstrate consistent performance to predict clinical deterioration and mortality in a large prospective second wave validation cohort of UK patients. Despite recent advances in the treatment and management of adults hospitalised with COVID-19, both scores can continue to inform clinical decision making. TRIAL REGISTRATION NUMBER: ISRCTN66726260.

Clinical prediction models for mortality in patients with covid-19: external validation and individual participant data meta-analysis.

OBJECTIVE: To externally validate various prognostic models and scoring rules for predicting short term mortality in patients admitted to hospital for covid-19. DESIGN: Two stage individual participant data meta-analysis. SETTING: Secondary and tertiary care. PARTICIPANTS: 46 914 patients across 18 countries, admitted to a hospital with polymerase chain reaction confirmed covid-19 from November 2019 to April 2021. DATA SOURCES: Multiple (clustered) cohorts in Brazil, Belgium, China, Czech Republic, Egypt, France, Iran, Israel, Italy, Mexico, Netherlands, Portugal, Russia, Saudi Arabia, Spain, Sweden, United Kingdom, and United States previously identified by a living systematic review of covid-19 prediction models published in The BMJ, and through PROSPERO, reference checking, and expert knowledge. MODEL SELECTION AND ELIGIBILITY CRITERIA: Prognostic models identified by the living systematic review and through contacting experts. A priori models were excluded that had a high risk of bias in the participant domain of PROBAST (prediction model study risk of bias assessment tool) or for which the applicability was deemed poor. METHODS: Eight prognostic models with diverse predictors were identified and validated. A two stage individual participant data meta-analysis was performed of the estimated model concordance (C) statistic, calibration slope, calibration-in-the-large, and observed to expected ratio (O:E) across the included clusters. MAIN OUTCOME MEASURES: 30 day mortality or in-hospital mortality. RESULTS: Datasets included 27 clusters from 18 different countries and contained data on 46 914patients. The pooled estimates ranged from 0.67 to 0.80 (C statistic), 0.22 to 1.22 (calibration slope), and 0.18 to 2.59 (O:E ratio) and were prone to substantial between study heterogeneity. The 4C Mortality Score by Knight et al (pooled C statistic 0.80, 95% confidence interval 0.75 to 0.84, 95% prediction interval 0.72 to 0.86) and clinical model by Wang et al (0.77, 0.73 to 0.80, 0.63 to 0.87) had the highest discriminative ability. On average, 29% fewer deaths were observed than predicted by the 4C Mortality Score (pooled O:E 0.71, 95% confidence interval 0.45 to 1.11, 95% prediction interval 0.21 to 2.39), 35% fewer than predicted by the Wang clinical model (0.65, 0.52 to 0.82, 0.23 to 1.89), and 4% fewer than predicted by Xie et al's model (0.96, 0.59 to 1.55, 0.21 to 4.28). CONCLUSION: The prognostic value of the included models varied greatly between the data sources. Although the Knight 4C Mortality Score and Wang clinical model appeared most promising, recalibration (intercept and slope updates) is needed before implementation in routine care.

Rapid synchronous type 1 IFN and virus-specific T cell responses characterize first wave non-severe SARS-CoV-2 infections.

Effective control of SARS-CoV-2 infection on primary exposure may reveal correlates of protective immunity to future variants, but we lack insights into immune responses before or at the time virus is first detected. We use blood transcriptomics, multiparameter flow cytometry, and T cell receptor (TCR) sequencing spanning the time of incident non-severe infection in unvaccinated virus-naive individuals to identify rapid type 1 interferon (IFN) responses common to other acute respiratory viruses and cell proliferation responses that discriminate SARS-CoV-2 from other viruses. These peak by the time the virus is first detected and sometimes precede virus detection. Cell proliferation is most evident in CD8 T cells and associated with specific expansion of SARS-CoV-2-reactive TCRs, in contrast to virus-specific antibodies, which lag by 1-2 weeks. Our data support a protective role for early type 1 IFN and CD8 T cell responses, with implications for development of universal T cell vaccines.

Blood transcriptional biomarkers of acute viral infection for detection of pre-symptomatic SARS-CoV-2 infection: a nested, case-control diagnostic accuracy study.

BACKGROUND: We hypothesised that host-response biomarkers of viral infections might contribute to early identification of individuals infected with SARS-CoV-2, which is critical to breaking the chains of transmission. We aimed to evaluate the diagnostic accuracy of existing candidate whole-blood transcriptomic signatures for viral infection to predict positivity of nasopharyngeal SARS-CoV-2 PCR testing. METHODS: We did a nested case-control diagnostic accuracy study among a prospective cohort of health-care workers (aged ≥18 years) at St Bartholomew's Hospital (London, UK) undergoing weekly blood and nasopharyngeal swab sampling for whole-blood RNA sequencing and SARS-CoV-2 PCR testing, when fit to attend work. We identified candidate blood transcriptomic signatures for viral infection through a systematic literature search. We searched MEDLINE for articles published between database inception and Oct 12, 2020, using comprehensive MeSH and keyword terms for "viral infection", "transcriptome", "biomarker", and "blood". We reconstructed signature scores in blood RNA sequencing data and evaluated their diagnostic accuracy for contemporaneous SARS-CoV-2 infection, compared with the gold standard of SARS-CoV-2 PCR testing, by quantifying the area under the receiver operating characteristic curve (AUROC), sensitivities, and specificities at a standardised Z score of at least 2 based on the distribution of signature scores in test-negative controls. We used pairwise DeLong tests compared with the most discriminating signature to identify the subset of best performing biomarkers. We evaluated associations between signature expression, viral load (using PCR cycle thresholds), and symptom status visually and using Spearman rank correlation. The primary outcome was the AUROC for discriminating between samples from participants who tested negative throughout the study (test-negative controls) and samples from participants with PCR-confirmed SARS-CoV-2 infection (test-positive participants) during their first week of PCR positivity. FINDINGS: We identified 20 candidate blood transcriptomic signatures of viral infection from 18 studies and evaluated their accuracy among 169 blood RNA samples from 96 participants over 24 weeks. Participants were recruited between March 23 and March 31, 2020. 114 samples were from 41 participants with SARS-CoV-2 infection, and 55 samples were from 55 test-negative controls. The median age of participants was 36 years (IQR 27-47) and 69 (72%) of 96 were women. Signatures had little overlap of component genes, but were mostly correlated as components of type I interferon responses. A single blood transcript for IFI27 provided the highest accuracy for discriminating between test-negative controls and test-positive individuals at the time of their first positive SARS-CoV-2 PCR result, with AUROC of 0·95 (95% CI 0·91-0·99), sensitivity 0·84 (0·70-0·93), and specificity 0·95 (0·85-0·98) at a predefined threshold (Z score >2). The transcript performed equally well in individuals with and without symptoms. Three other candidate signatures (including two to 48 transcripts) had statistically equivalent discrimination to IFI27 (AUROCs 0·91-0·95). INTERPRETATION: Our findings support further urgent evaluation and development of blood IFI27 transcripts as a biomarker for early phase SARS-CoV-2 infection for screening individuals at high risk of infection, such as contacts of index cases, to facilitate early case isolation and early use of antiviral treatments as they emerge. FUNDING: Barts Charity, Wellcome Trust, and National Institute of Health Research.

Discovery and validation of a three-gene signature to distinguish COVID-19 and other viral infections in emergency infectious disease presentations: a case-control and observational cohort study.

BACKGROUND: Emergency admissions for infection often lack initial diagnostic certainty. COVID-19 has highlighted a need for novel diagnostic approaches to indicate likelihood of viral infection in a pandemic setting. We aimed to derive and validate a blood transcriptional signature to detect viral infections, including COVID-19, among adults with suspected infection who presented to the emergency department. METHODS: Individuals (aged ≥18 years) presenting with suspected infection to an emergency department at a major teaching hospital in the UK were prospectively recruited as part of the Bioresource in Adult Infectious Diseases (BioAID) discovery cohort. Whole-blood RNA sequencing was done on samples from participants with subsequently confirmed viral, bacterial, or no infection diagnoses. Differentially expressed host genes that met additional filtering criteria were subjected to feature selection to derive the most parsimonious discriminating signature. We validated the signature via RT-qPCR in a prospective validation cohort of participants who presented to an emergency department with undifferentiated fever, and a second case-control validation cohort of emergency department participants with PCR-positive COVID-19 or bacterial infection. We assessed signature performance by calculating the area under receiver operating characteristic curves (AUROCs), sensitivities, and specificities. FINDINGS: A three-gene transcript signature, comprising HERC6, IGF1R, and NAGK, was derived from the discovery cohort of 56 participants with bacterial infections and 27 with viral infections. In the validation cohort of 200 participants, the signature differentiated bacterial from viral infections with an AUROC of 0·976 (95% CI 0·919-1·000), sensitivity of 97·3% (85·8-99·9), and specificity of 100% (63·1-100). The AUROC for C-reactive protein (CRP) was 0·833 (0·694-0·944) and for leukocyte count was 0·938 (0·840-0·986). The signature achieved higher net benefit in decision curve analysis than either CRP or leukocyte count for discriminating viral infections from all other infections. In the second validation analysis, which included SARS-CoV-2-positive participants, the signature discriminated 35 bacterial infections from 34 SARS-CoV-2-positive COVID-19 infections with AUROC of 0·953 (0·893-0·992), sensitivity 88·6%, and specificity of 94·1%. INTERPRETATION: This novel three-gene signature discriminates viral infections, including COVID-19, from other emergency infection presentations in adults, outperforming both leukocyte count and CRP, thus potentially providing substantial clinical utility in managing acute presentations with infection. FUNDING: National Institute for Health Research, Medical Research Council, Wellcome Trust, and EU-FP7.

Development and validation of the ISARIC 4C Deterioration model for adults hospitalised with COVID-19: a prospective cohort study.

BACKGROUND: Prognostic models to predict the risk of clinical deterioration in acute COVID-19 cases are urgently required to inform clinical management decisions. METHODS: We developed and validated a multivariable logistic regression model for in-hospital clinical deterioration (defined as any requirement of ventilatory support or critical care, or death) among consecutively hospitalised adults with highly suspected or confirmed COVID-19 who were prospectively recruited to the International Severe Acute Respiratory and Emerging Infections Consortium Coronavirus Clinical Characterisation Consortium (ISARIC4C) study across 260 hospitals in England, Scotland, and Wales. Candidate predictors that were specified a priori were considered for inclusion in the model on the basis of previous prognostic scores and emerging literature describing routinely measured biomarkers associated with COVID-19 prognosis. We used internal-external cross-validation to evaluate discrimination, calibration, and clinical utility across eight National Health Service (NHS) regions in the development cohort. We further validated the final model in held-out data from an additional NHS region (London). FINDINGS: 74 944 participants (recruited between Feb 6 and Aug 26, 2020) were included, of whom 31 924 (43·2%) of 73 948 with available outcomes met the composite clinical deterioration outcome. In internal-external cross-validation in the development cohort of 66 705 participants, the selected model (comprising 11 predictors routinely measured at the point of hospital admission) showed consistent discrimination, calibration, and clinical utility across all eight NHS regions. In held-out data from London (n=8239), the model showed a similarly consistent performance (C-statistic 0·77 [95% CI 0·76 to 0·78]; calibration-in-the-large 0·00 [-0·05 to 0·05]); calibration slope 0·96 [0·91 to 1·01]), and greater net benefit than any other reproducible prognostic model. INTERPRETATION: The 4C Deterioration model has strong potential for clinical utility and generalisability to predict clinical deterioration and inform decision making among adults hospitalised with COVID-19. FUNDING: National Institute for Health Research (NIHR), UK Medical Research Council, Wellcome Trust, Department for International Development, Bill & Melinda Gates Foundation, EU Platform for European Preparedness Against (Re-)emerging Epidemics, NIHR Health Protection Research Unit (HPRU) in Emerging and Zoonotic Infections at University of Liverpool, NIHR HPRU in Respiratory Infections at Imperial College London.

Time series analysis and mechanistic modelling of heterogeneity and sero-reversion in antibody responses to mild SARS­CoV-2 infection.

BACKGROUND: SARS-CoV-2 serology is used to identify prior infection at individual and at population level. Extended longitudinal studies with multi-timepoint sampling to evaluate dynamic changes in antibody levels are required to identify the time horizon in which these applications of serology are valid, and to explore the longevity of protective humoral immunity. METHODS: Healthcare workers were recruited to a prospective cohort study from the first SARS-CoV-2 epidemic peak in London, undergoing weekly symptom screen, viral PCR and blood sampling over 16-21 weeks. Serological analysis (n =12,990) was performed using semi-quantitative Euroimmun IgG to viral spike S1 domain and Roche total antibody to viral nucleocapsid protein (NP) assays. Comparisons were made to pseudovirus neutralizing antibody measurements. FINDINGS: A total of 157/729 (21.5%) participants developed positive SARS-CoV-2 serology by one or other assay, of whom 31.0% were asymptomatic and there were no deaths. Peak Euroimmun anti-S1 and Roche anti-NP measurements correlated (r = 0.57, p<0.0001) but only anti-S1 measurements correlated with near-contemporary pseudovirus neutralising antibody titres (measured at 16-18 weeks, r = 0.57, p<0.0001). By 21 weeks' follow-up, 31/143 (21.7%) anti-S1 and 6/150 (4.0%) anti-NP measurements reverted to negative. Mathematical modelling revealed faster clearance of anti-S1 compared to anti-NP (median half-life of 2.5 weeks versus 4.0 weeks), earlier transition to lower levels of antibody production (median of 8 versus 13 weeks), and greater reductions in relative antibody production rate after the transition (median of 35% versus 50%). INTERPRETATION: Mild SARS-CoV-2 infection is associated with heterogeneous serological responses in Euroimmun anti-S1 and Roche anti-NP assays. Anti-S1 responses showed faster rates of clearance, more rapid transition from high to low level production rate and greater reduction in production rate after this transition. In mild infection, anti-S1 serology alone may underestimate incident infections. The mechanisms that underpin faster clearance and lower rates of sustained anti-S1 production may impact on the longevity of humoral immunity. FUNDING: Charitable donations via Barts Charity, Wellcome Trust, NIHR.

Evaluation and improvement of the National Early Warning Score (NEWS2) for COVID-19: a multi-hospital study.

BACKGROUND: The National Early Warning Score (NEWS2) is currently recommended in the UK for the risk stratification of COVID-19 patients, but little is known about its ability to detect severe cases. We aimed to evaluate NEWS2 for the prediction of severe COVID-19 outcome and identify and validate a set of blood and physiological parameters routinely collected at hospital admission to improve upon the use of NEWS2 alone for medium-term risk stratification. METHODS: Training cohorts comprised 1276 patients admitted to King's College Hospital National Health Service (NHS) Foundation Trust with COVID-19 disease from 1 March to 30 April 2020. External validation cohorts included 6237 patients from five UK NHS Trusts (Guy's and St Thomas' Hospitals, University Hospitals Southampton, University Hospitals Bristol and Weston NHS Foundation Trust, University College London Hospitals, University Hospitals Birmingham), one hospital in Norway (Oslo University Hospital), and two hospitals in Wuhan, China (Wuhan Sixth Hospital and Taikang Tongji Hospital). The outcome was severe COVID-19 disease (transfer to intensive care unit (ICU) or death) at 14 days after hospital admission. Age, physiological measures, blood biomarkers, sex, ethnicity, and comorbidities (hypertension, diabetes, cardiovascular, respiratory and kidney diseases) measured at hospital admission were considered in the models. RESULTS: A baseline model of 'NEWS2 + age' had poor-to-moderate discrimination for severe COVID-19 infection at 14 days (area under receiver operating characteristic curve (AUC) in training cohort = 0.700, 95% confidence interval (CI) 0.680, 0.722; Brier score = 0.192, 95% CI 0.186, 0.197). A supplemented model adding eight routinely collected blood and physiological parameters (supplemental oxygen flow rate, urea, age, oxygen saturation, C-reactive protein, estimated glomerular filtration rate, neutrophil count, neutrophil/lymphocyte ratio) improved discrimination (AUC = 0.735; 95% CI 0.715, 0.757), and these improvements were replicated across seven UK and non-UK sites. However, there was evidence of miscalibration with the model tending to underestimate risks in most sites. CONCLUSIONS: NEWS2 score had poor-to-moderate discrimination for medium-term COVID-19 outcome which raises questions about its use as a screening tool at hospital admission. Risk stratification was improved by including readily available blood and physiological parameters measured at hospital admission, but there was evidence of miscalibration in external sites. This highlights the need for a better understanding of the use of early warning scores for COVID.

Systematic evaluation and external validation of 22 prognostic models among hospitalised adults with COVID-19: an observational cohort study.

The number of proposed prognostic models for coronavirus disease 2019 (COVID-19) is growing rapidly, but it is unknown whether any are suitable for widespread clinical implementation.We independently externally validated the performance of candidate prognostic models, identified through a living systematic review, among consecutive adults admitted to hospital with a final diagnosis of COVID-19. We reconstructed candidate models as per original descriptions and evaluated performance for their original intended outcomes using predictors measured at the time of admission. We assessed discrimination, calibration and net benefit, compared to the default strategies of treating all and no patients, and against the most discriminating predictors in univariable analyses.We tested 22 candidate prognostic models among 411 participants with COVID-19, of whom 180 (43.8%) and 115 (28.0%) met the endpoints of clinical deterioration and mortality, respectively. Highest areas under receiver operating characteristic (AUROC) curves were achieved by the NEWS2 score for prediction of deterioration over 24 h (0.78, 95% CI 0.73-0.83), and a novel model for prediction of deterioration <14 days from admission (0.78, 95% CI 0.74-0.82). The most discriminating univariable predictors were admission oxygen saturation on room air for in-hospital deterioration (AUROC 0.76, 95% CI 0.71-0.81), and age for in-hospital mortality (AUROC 0.76, 95% CI 0.71-0.81). No prognostic model demonstrated consistently higher net benefit than these univariable predictors, across a range of threshold probabilities.Admission oxygen saturation on room air and patient age are strong predictors of deterioration and mortality among hospitalised adults with COVID-19, respectively. None of the prognostic models evaluated here offered incremental value for patient stratification to these univariable predictors.